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(57) In one aspect, a back-channel communication 
network for a videoconferencing system for a confer- 
ence between a plurality of participants is provided. The 
back-channel communication network includes a mon- 
itoring agent associated with a client. The client is con- 
figured to execute a peer-to-peer videoconferencing ap- 
plication. The monitoring agent monitoring a video dis- 
play window is control led by the peer-to-peer conferenc- 
ing application. A back-channel controller in communi- 
cation with the monitoring agent over a back-channel 
connection is included. The back-channel controller is 
configured to enable communication between the client 
and a plurality of conference clients over a back-channel 
controller communication link. An event handler config- 
ured to enable insertion of server user interface data into 
an outbound video stream image for the client is also 
included. A computer readable media and methods for 
providing a multi-participant conferencing environment 
are also provided. In another aspect a videoconferenc- 
ing system configured to utilize peer-to-peer videocon- 
ferencing software to provide a multi-participant confer- 
ence environment for a plurality of participants is pro- 
vided. The system includes a client component defining 
a conference client enabled to execute peer-to-peer vid- 
eoconferencing software. The client component in- 
cludes a client monitor configured to monitor both, 
whether the conference channel is active and events 
within a video window displayed by the conference cli- 



ent, wherein the events within the video window are 
communicated across a back-channel connection. The 
back-channel connection is established when the con- 
ference channel is active. The system includes a server 
component having a back-channel controller in commu- 
nication with the client monitor through the back-chan- 
nel connection. The server component provides a client 
configurable audio/video stream for each of a plurality 
of participants. A graphical user interface and methods 
for providing a multi-participant conferencing environ- 
ment are provided. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001] This invention relates generally to videoconfer- 
encing systems and more particularly to a system capa- 
ble of utilizing pre-existing peer-to-peer videoconfer- 
encing applications and a multi-point control unit (MCU) 
managed by a participant-controllable content delivery 
interface. 

2. Description of the Related Art 

[0002] Conferencing devices are used to facilitate 
communication between two or more participants phys- 
ically located at separate locations. Devices are availa- 
ble to exchange live video, audio, and other data to view, 
hear, or otherwise collaborate with each participant. 
Common applications for conferencing include meet- 
ings/workgroups, presentations, and training/educa- 
tion. Today, with the help of videoconferencing software, 
a personal computer with an inexpensive camera and 
microphone can be used to connect with other confer- 
encing participants. The operating systems of some of 
these machines provide simple peer-to-peer videocon- 
ferencing software, such as MICROSOFT'S NETMEET- 
ING application that is included with MICROSOFT WIN- 
DOWS based operating systems. Alternatively, peer-to- 
peer videoconferencing software application can be in- 
expensively purchased separately. Motivated by the 
availability of software and inexpensive camera/micro- 
phone devices, videoconferencing has become increas- 
ingly popular. 

[0003] Video communication relies on sufficiently 
large and fast networks to accommodate the high infor- 
mation content of moving images. Audio and video data 
communication also demand adequate bandwidth as 
the number of participants and the size of the data ex- 
change increase. Even with compression technologies 
and limitations in content size, sufficient bandwidth for 
multi-party conferences is not readily available using 
common and inexpensive transport systems. 
[0004] Figures 1A-1C illustrate the content transfer 
requirements for each participant in a two, three, or four 
member conference, respectively. As can be seen, each 
member must send and receive content from each of 
the other participants. As the number of participants in- 
crease, so too does the connection requirements for 
each participant. For example, where there are two par- 
ticipants each participant requires two connections, 
where there are three participants each participant re- 
quires four connections, where there are four partici- 
pants each participant requires six connections, and so 
on. As a consequence of the increased connection re- 
quirements, the systems supporting these requirements 
become more sophisticated and of course, more expen- 



sive. Thus, most inexpensive videoconferencing sys- 
tems iimit a participant to connecting with only one other 
member, i.e. a peer-to-peer connection. 
[0005] Devices are available to address the excessive 

5 amount of connections. A Multipoint Control Unit (MCU) 
helps resolve the connection issue by establishing a 
central location for connection by all participants. An 
MCU is an external device that efficiently allows three 
or more participants to establish a shared conference. 

10 A peer-to-peer connection is established between the 
MCU and each conference participant using the partic- 
ipant videoconference software. Figures 2A-2C illus- 
trates the connection reduction offered by a MCU as 
compared to the connection requirements of Figures 

15 1A-1C. In particular, for two participants, each partici- 
pant has two connections, for three participants, each 
participant has three connections, for four participants, 
each participant has four connections, and so on. While 
the MCU reduces the amount of outgoing connections 

20 each participant must manage, the incoming content 
transfer requirements are still too high to manage large 
conferences. 

[0006] An MCU can offload more processing from the 
participant's machine by reducing the content it sends 
25 to each participant. For example, an MCU can choose 
to send only the content of the participant who is speak- 
ing. Alternately, the MCU can choose to combine par- 
ticipant audio and video signals. When combining video, 
signal loss will occur as each participant's video signal 
30 is scaled to a smaller fraction of its original size. Often 
MCUs will combine only the audio signals so that all 
members can be heard, and send only the video signal 
of the active speaker. By using these offloading tech- 
niques, less information needs to be transferred to each 

35 participant. 

[0007] A shortcoming of the MCU is the lack of flexi- 
bility allowed for the conference participants. That is, 
there is a small fixed set of configuration features offered 
to the participants. In addition, the MCU is often man- 

40 aged by a remote administrator that further limits any 
dynamic configuration of the conference presentation 
by the participants. Yet another, limitation in using peer- 
to-peer software with the MCU is that the peer-to-peer 
software is not designed to provide features for a multi- 

45 participant conference environment. More particularly, 
the peer-to-peer software applications, whether includ- 
ed with an operating system or purchased separately, 
is limited to features provided exclusively for peer-to- 
peer conferencing environments. 

so [0008] As a result, there is a need to solve the prob- 
lems of the prior art to provide a method and apparatus 
for enabling a multi-participant videoconferencing envi- 
ronment where the participants have peer-to-peer vide- 
oconferencing software such that the videoconferenc- 

55 ing environment allows the user flexibility in defining 
configuration features and content delivery. 
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SUMMARY OF THE INVENTION 

[0009] Broadly speaking, the present invention fills 
these needs by providing a method and system for pro- 
viding a multi participant videoconferencing environ- 
ment with clients having pre-existing peer-to-peer vide- 
oconferencing applications. A back-channel connection 
is provided to allow participant customizable video lay- 
outs to be displayed for each participant. Additionally, 
the audio distribution is customizable through informa- 
tion provided over the back-channel. It should be appre- 
ciated that the present invention can be implemented in 
numerous ways, including as a process, a system, or a 
graphical user interface. Several inventive embodi- 
ments of the present invention are described below. 
[0010] In one embodiment, a videoconference sys- 
tem is provided. The videoconference system includes 
a client component having a monitoring agent config- 
ured to detect events within a video display window of 
the client component. A server component configured 
to distribute video and audio data streams to partici- 
pants of a conference session is included. A conference 
channel communication connection over which the vid- 
eo and audio data streams are carried between the cli- 
ent component and the server component is provided. 
A back-channel communication connection over which 
events captured by the monitoring agent are transmitted 
to the server component is included. The back-channel 
communication connection enables each of the partici- 
pants to define a video layout of the video display win- 
dow. 

[0011] In another embodiment, a back-channel com- 
munication network for a videoconferencing system for 
a conference between a plurality of participants is pro- 
vided. The back-channel communication network in- 
cludes a monitoring agent associated with a client. The 
client is configured to execute a peer-to-peer videocon- 
ferencing application. The monitoring agent monitoring 
a video display window controlled by the peer-to-peer 
conferencing application. A back-channel controller in 
communication with the monitoring agent over a back- 
channel connection is included. The back-channel con- 
troller is configured to enable communication between 
the client and a plurality of conference clients over a 
back-channel controller communication link. An event 
handler configured to enable insertion of server user in- 
terface data into an outbound video stream image for 
the client is also included. 

[0012] In yet another embodiment, a method for en- 
hancing conference content delivery for a videoconfer- 
ence session between multiple participants is provided. 
The method initiates with monitoring a video display win- 
dow associated with a client. Next, a conference chan- 
nel connection is established for transmitting a video 
stream and an audio stream between the client and a 
server. Then, the establishment of the conference chan- 
nel connection is detected, in response to detecting the 
conference channel connection, the method includes 
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establishing a back-channel connection between the cli- 
ent and the server. Then, a server user-interface (SUI) 
is inserted into the video stream. Next, the video stream 
is displayed in the video display window of the client. 
5 Then, an active selection is detected in an active region 
of the video display window. Next, the active selection 
is communicated to the server over the back-channel 
connection. Then, a configuration of one of the video 
stream and the audio stream is modified at the server. 
10 Next, the modified configuration is provided to the client 
over the conference channel connection. 
[0013] In still yet another embodiment, a method for 
providing participant customizable video and audio 
streams for a videoconference session between a plu- 
15 rality of participants is provided. The method initiates 
with providing a plurality of clients, each of the plurality 
of clients associated with a participant. Then, a server 
in communication with the plurality of clients is provided. 
Next, a first communication channel and second com- 
20 munication channel are established between the server 
and each of the plurality of clients. The first communi- 
cation channel provides audio/video data. The second 
communication channel provides system information. 
Then, a video display window of a client is monitored. 
25 Next, feedback from the monitoring of the video display 
window is provided over the second communication 
channel to modify the audio/video data being supplied 
over the first communication channel. 
[0014] In still yet another embodiment, a computer 
30 readable media having program instructions for provid- 
ing participant customizable video and audio streams 
for a videoconference session between a plurality of 
participants is provided. The computer readable media 
includes program instructions for providing a plurality of 
35 clients where each of the plurality of clients is associated 
with a participant. Program instructions for providing a 
server in communication with the plurality of clients are 
included. Program instructions for establishing a first 
communication channel and second communication 
40 channel between the server and each of the plurality of 
clients are provided. The first communication channel 
provides audio/video data, while the second communi- 
cation channel provides system information. Program 
instructions for monitoring a video display window of a 
45 client are included. Program instructions for providing 
feedback from the monitoring of the display window over 
the second communication channel to modify the audio/ 
video data being supplied over the first communication 
channel are also provided. 
50 [0015] In a further embodiment, a videoconferencing 
system configured to utilize peer-to-peer videoconfer- 
encing software to provide a multi-participant confer- 
ence environment for a plurality of participants is pro- 
vided. The system includes a client component. The cli- 
55 ent component includes a conference client enabled to 
execute peer-to-peer videoconferencing software. The 
conference client communicates video and audio data 
across a conference channel. The client component in- 



EP 1 381 237 A2 



5 



EP 1 381 237 A2 



6 



eludes a client monitor configured to monitor both, 
whether the conference channel is active and events 
within a video window displayed by the conference cli- 
ent, wherein the events within the video window are 
communicated across a back-channel connection. The 
back-channel connection is established when the con- 
ference channel is active. The system includes a server 
component, the server component having a back-chan- 
nel controller in communication with the client monitor 
through the back-channel connection. The server com- 
ponent provides a client configurable video stream for 
each of a plurality of participants. 
[0016] In another embodiment, a videoconferencing 
system is provided. The videoconferencing system in- 
cludes a client component having a client in communi- 
cation with a client monitor. The videoconferencing sys- 
tem includes a server component. A conference chan- 
nel defined between the client component and the serv- 
er component is included. The conference channel pro- 
vides a first path for real-time video/audio data to be ex- 
changed between the client component and a confer- 
encing endpoint of the server component for a video- 
conference session. A back-channel defined between 
the client component and the server component is in- 
cluded. The back-channel provides a second path for 
system information to be exchanged between the client 
monitor and the server component. 
[0017] In yet another embodiment, a conferencing 
system configured to provide a multi-user conference 
environment to deliver customizable information to a 
plurality of participants is provided. The conferencing 
system includes a client component. The client compo- 
nent includes a conference client. A client monitor is in- 
cluded in the client component. The client monitor is 
configured to monitor an activity of the conference client, 
wherein the activity occurs over a video frame displayed 
by the conference client. The conferencing system in- 
cludes a server component. The server component in- 
cludes a media hub server component providing a con- 
ference connection. The media hub server component 
includes a media mixer that is configured to assemble 
audio and video data to be supplied to the conference 
client from audio and video data received by the media 
mixer from a plurality of conference clients. The media 
mixer includes a video layout processor configured to 
generate a composite video image for each of the plu- 
rality of conference clients. The media mixer also in- 
cludes an audio distribution processor for providing an 
audio signal for each of the plurality of conference cli- 
ents. The server component includes a connection man- 
ager allowing connections of several participants into 
logical rooms for shared conference communications. 
The connection manager includes a back-channel con- 
troller enabling communication between the client mon- 
itor and the media hub server component. The connec- 
tion manager also includes an event handler configured 
to insert interface data into an outbound video stream 
image through the video layout processor. 



[0018] In still yet another embodiment, a graphical us- 
er interface (GUI) for a videoconference rendered on a 
computer monitor is provided. The GUI includes a first 
region defining an integrated video component. The in- 
5 tegrated video component is associated with a client. 
The integrated video component has a plurality of par- 
ticipant video images. The integrated video component 
is monitored to detect user activity within a display win- 
dow of the integrated video component. The GUI in- 
fo eludes a second region providing access to files of a 
computer system. The second region allows a user to 
select one of the files for transmission to a server sup- 
porting the videoconference, wherein the server com- 
municates the selected one of the files to participants of 
the videoconference. 

[0019] In another embodiment, a method for providing 
a multi-user conference environment for multiple partic- 
ipants is provided. The method initiates with establish- 
ing a server component for enabling a conference chan- 

20 nel connection between the server component and a 
conference client associated with a participant. Then, 
audio and video data from the participant is provided to 
the server component overthe conference channel con- 
nection. Next, system preferences are communicated 

25 to the server component for each of the multiple clients 
over a back-channel connection. Then, combined audio 
and video data is distributed to the participant overthe 
conference channel connection. The combined audio 
and video data is presented as defined by the system 

30 preferences. Next, an interaction of the participant with 
a video image presented on the conference client is 
monitored. Then, a signal indicating the interaction to 
the server component is transmitted over the back- 
channel connection. In response to the signal indicating 

35 the interaction, the combined audio and video data is 
modified and distributed to the conference client over 
the conference channel connection. 
[0020] In yet another embodiment, a method for cre- 
ating a multi-user conferencing environment between 

40 conference clients having peer-to-peer conferencing 
applications is provided. The method initiates with pro- 
viding a server component configured to emulate a 
peer-to-peer connection for each of the conference cli- 
ents. Then, a conference channel is defined for commu- 

45 nication between conference clients and the server 
component. Next, activities of a user in an active region 
of a video display associated with one of the conference 
clients are monitored. Then, an active selection by a us- 
er in the active region is reported to the server compo- 

50 nent. The reporting of the active selection occurs out- 
side of the conference channel. In response to the active 
selection reporting being received by the server compo- 
nent, a configuration of an audio/video signal is modified 
and provided to the conference clients. 

55 [0021] In still yet another embodiment, a computer 
readable media having program instructions for creating 
a multi-user conferencing environment between confer- 
ence clients having peer-to-peer conferencing applica- 
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tions and a server component configured to emulate a 
peer-to-peer connection for each of the participants is 
provided. The computer readable media includes pro- 
gram instructions for defining a conference channel for 
communication between conference clients and the 
server component. Program instructions for monitoring 
activities of a user with one of the conference clients are 
included. Program instructions for reporting the moni- 
tored activities to the server component over a back- 
channel connection are included. Program instructions 
for modifying a video and audio signal provided to the 
conference clients in response to the reported activities 
being received by the server component are also includ- 
ed. 

[0022] Other aspects and advantages of the invention 
will become apparent from the following detailed de- 
scription, taken in conjunction with the accompanying 
drawings, illustrating by way of example the principles 
of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0023] The present invention will be readily under- 
stood by the following detailed description in conjunction 
with the accompanying drawings, and like reference nu- 
merals designate like structural elements. 
[0024] Figures 1A-1C illustrate the content transfer 
requirements for each participant in a two, three, or four 
member conference, respectively. 
[0025] Figures 2A-2C illustrates the connection re- 
duction offered by a MCU as compared to the intercon- 
nections of Figures 1 A-1C. 

[0026] Figure 3 is a simplified schematic diagram of 
a high level overview of a videoconferencing system 
having a back-channel communication link in accord- 
ance with one embodiment of the invention. 
[0027] Figure 4 is a schematic diagram of the compo- 
nents for a multi-participant conference system using a 
client monitor back-channel in accordance with one em- 
bodiment of the invention. 

[0028] Figure 5 is a schematic diagram of the compo- 
nents for a multi-participant conference system using a 
client monitor back-channel wherein a non-participant 
can join the conference in accordance with one embod- 
iment of the invention. 

[0029] Figure 6 is a high level schematic diagram of 
the media hub server in accordance with one embodi- 
ment of the invention. 

[0030] Figure 7 is a more detailed schematic diagram 
of the client monitor connection between the client and 
the media hub server in accordance with one embodi- 
ment of the invention. 

[0031 ] Figure 8 is a schematic diagram of a video lay- 
out processor configured to generate a composite video 
image for each participant in accordance with one em- 
bodiment of the invention. 

[0032] Figure 9 is a schematic diagram of the audio 
distribution processor in accordance with one embodi- 
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ment of the invention. 

[0033] Figure 10 is a schematic diagram of the audio 
distribution processor configured to provide private au- 
dio communications in accordance with one embodi- 

5 ment of the invention. 

[0034] Figures 11A-11C are schematic diagrams of 
patterns for mixing audio streams in accordance with 
one embodiment of the invention. 
[0035] Figure 12 is a schematic diagram of the effect 

10 of an event on a conference client's video display win- 
dow in accordance with one embodiment of the inven- 
tion. 

[0036] Figure 13 is a schematic diagram of another 
effect of an event on a conference client's video display 
15 window in accordance with one embodiment of the in- 
vention. 

[0037] Figure 14 is a schematic diagram of a client 
monitor graphical user interface which includes the user 
interface provided by the conference client in accord- 

20 ance with one embodiment of the invention. 

[0038] Figure 1 5 is a flowchart diagram of the method 
operations for creating a multi-user conferencing envi- 
ronment between conference clients having peer-to- 
peer conferencing applications in accordance with one 

25 embodiment of the invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

30 [0039] An invention is described for an apparatus and 
method for a videoconferencing system having a 
multipoint controller configured to mix audio/video 
streams from multiple participants into a single audio/ 
video stream. The multipoint controller is configured to 
35 provide server constructed interface elements into the 
audio/video stream based upon client monitored events. 
It will be obvious, however, to one skilled in the art. that 
the present invention may be practiced without some or 
all of these specific details. In other instances, well 
40 known process operations have not been described in 
detail in order not to unnecessarily obscure the present 
invention. Figures 1A-1C and 2A-2C are described in 
the "Background of the Invention" section. 
[0040] The embodiments of the present invention pro- 
45 vide a method and apparatus for providing a multi-user 
conferencing environment. The multi-user conferencing 
environment includes a multi-point control unit enabled 
to provide multi-participant features while connecting 
clients having pre-existing peer-to-peer videoconfer- 
50 encing software. The conferencing system includes a 
parallel connection to the conference channel that al- 
lows for the ability to define functionality through a client 
monitor that watches the participant's interactions with 
the pre-existing videoconferencing software's. In one 
55 embodiment, the participant's interactions that occur in 
a window displaying the video stream are monitored. In 
effect, the client monitor acts similarly to a conference 
user, with respect to watching the pre-existing videocon- 
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ferencing software's video stream. It should be appreci- 
ated that the code defining the client monitor executes 
externally to the conference client, i.e., the client monitor 
code is separate and distinct from the conference client 
software. As used herein, the terms client monitor and 
external client monitor are interchangeable. 
[0041] The videoconferencing system includes a cli- 
ent component and a server component. The client 
component includes a client monitor and a conference 
client. The client monitor captures input from the con- 
ference client. In one embodiment, the conference client 
is a peer-to-peer videoconferencing application. One 
example of a peer-to-peer videoconferencing applica- 
tion is MICROSOFT'S NETMEETING application. How- 
ever, one skilled in the art will appreciate that any peer- 
to-peer videoconferencing application is suitable for the 
embodiments described herein. Thus, the system en- 
hances pre-existing applications, which may already be 
installed on a personal computer, with increased func- 
tionality enabled through data provided by the client 
monitor. In addition, the client monitor can incorporate 
a graphical user interface (GUI) in which the video win- 
dow of the peer-to-peer application is a component. 
[0042] The client monitor provides the captured input 
from the conference client to a server component. The 
captured input is transmitted to the server component 
through a separate connection, i.e., a back-channel 
connection, that operates in parallel with the existing 
conference client's conference channel. In one embod- 
iment, the back-channel system enables the server to 
dynamically modify the GUI being presented to a partic- 
ipant based on the captured input provided to the server 
component. For example, the client monitor can capture 
events, such as mouse clicks or mouse clicks in combi- 
nation with keyboard strokes, executed by a user when 
his mouse pointer is within a region of the conference 
clientthat displays the video signal. In one embodiment, 
the events are transmitted through a back-channel con- 
nection to the server component for interpretation. Thus, 
the back-channel connection allows for active regions 
and user interface objects within the video stream to be 
used to control functionality and content. Consequently, 
users, i.e., also referred to as participants herein, indi- 
rectly control video given to different regions in the lay- 
out based upon server processing of client events. As 
will be described below, additional communication ex- 
change is available between participants using this sys- 
tem's back-channel connection. 
[0043] Figure 3 is a simplified schematic diagram of 
a high level overview of a videoconferencing system 
having a back-channel communication link in accord- 
ance with one embodiment of the invention. Hub and 
mixer 120 represent the server side component of the 
videoconferencing system. Participants P1 122a 
through Pn 122n represent the client component of the 
videoconferencing system. Each of the participants in- 
terface with server component 1 20 through two commu- 
nication links. Communication link 124 is a conference 
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channel providing real time audio and video signals be- 
tween the client component and server component 1 20. 
One skilled in the art will appreciate that conference 
channels 124a-124n can support any suitable stand- 

5 ards for use on packet switched Internet Protocol (IP) 
networks, such as H.323 standards, Session Initiation 
Protocol (SIP) standards, etc. Back-channel connection 
1 26 is a communication link that allows input, i.e., events 
captured from the video display region or a client mon- 

10 itor graphical user interface (GUI) of client component 
122, to be transmitted to server component 120. 
[0044] Figure 4 is a schematic diagram of the compo- 
nents for a multi-participant conference system using a 
client monitor back-channel in accordance with one em- 

15 bodiment of the invention. The client component in- 
cludes multiple participants, such as participant A 122a 
through participant N 122n. Each participant 122 in- 
cludes conference client 1 44 and client monitor 1 46. For 
example, participant A 122a includes conference client 

20 a 144a and client monitor A 146a. In one embodiment, 
conference client A 1 44a includes the participant's peer- 
to-peer videoconferencing software. The role of confer- 
ence client A is to place calls to another participant, es- 
tablish and disconnect a conferencing session, capture 

25 and send content, receive and playback the content ex- 
changed, etc. It should be appreciated that calls from 
conference client A 1 44a route through media hub serv- 
er 1 30. Other participants similarly use their associated 
conference client to place calls to media hub server 1 30 

30 to join the conference. In one embodiment, conference 
client A 144a includes a high-level user-interface for the 
conference, such as when the conference client is a pre- 
existing software application. For example, a product 
that provides peer-to-peer videoconferencing is the 

35 NETMEETING application software from MICROSOFT 
Corporation. 

[0045] Client monitor (CM) 146 is monitoring confer- 
ence client 144. CM 146a is configured to monitor con- 
ference client A 144a. That is, CM 146a looks at how a 

40 user is interacting with the software application by mon- 
itoring a video display window of client A 144a in one 
embodiment. In addition, CM 146a interprets the users 
interactions in order to transmit the interactions to the 
server component. In one embodiment, CM 1 46 is con- 

45 figured to provide four functions. One function monitors 
the start/stop of a conference channel so that a back- 
channel communication session can be established in 
parallel to a conference channel session between the 
participant and the server component. A second func- 

50 tion monitors events, such as user interactions and 
mouse messages, within the video window displayed by 
conference client 144. A third function handles control 
message information between the CM 146 and a back- 
channel controller 140 of the server component. A fourth 

55 function provides an external user-interface for the par- 
ticipant that can be used to display and send images to 
other conference members, show the other connected 
participants names, and other communication informa- 
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tion or tools as described in more detail with reference 
to Figure 14. 

[0046] As mentioned above, client monitor 146 watch- 
es for activity in conference client 144. In one embodi- 
ment, this includes monitoring user events over the vid- 
eo display region containing the conference content, 
and also includes the conference session control infor- 
mation. For example, CM 1 46 watches for the start and 
end of a conference session or a call from the confer- 
ence client. When conference client 1 44 places a call to 
media hub server 1 30 to start a new conference session, 
CM 146 also places a call to the media hub server. The 
call from CM 146 establishes back-channel connection 
126 for the participant's conference session. Since CM 
146 can monitor the session start/stop events, back- 
channel connection initiates automatically without addi- 
tional usersetup : i.e., the back-channel connection is 
transparent to a user. Accordingly, a new session is 
maintained in parallel with conference client 1 44 activity. 
It should be appreciated that conference channel 124 
provides a video/audio connection between conference 
client 1 44 and conference connection 1 38 of media hub 
server 130. In one embodiment, conference channel 
124 provides a communication link for real time video/ 
audio data of the conference session communicated be- 
tween the client component and the server component. 
[0047] In one embodiment, CM 146 specifically mon- 
itors activity that occurs over the conference's video 
frame displayed by conference client 1 44. For example, 
CM 146 may monitor the video image in MICROSOFTS 
NETMEETING application. Mouse activity in the client 
frame is relayed via protocol across back-channel con- 
nection 126to media hub server 130. In turn, back-chan- 
nel controller 140 can report this activity to another par- 
ticipant, or event handler 142 for the respective partici- 
pant. In this embodiment, the monitoring of conference 
client 144 application occurs through a hook between 
the operating system level and the application level. As 
mentioned above, the video window can be watched for 
mouse clicks or keyboard strokes from outside of the 
videoconferencing application. 
[0048] In another embodiment, CM 146 can present 
a separate user-interface to the participant. This inter- 
face can be shown in parallel to the user interface pre- 
sented by conference client 144 and may remain 
throughout the established conference. Alternatively, 
the user interface presented by CM 1 46 may appear be- 
fore or after a conference session for other configuration 
or setup purposes. One embodiment of the user inter- 
face is illustrated in Figure 14. 

[0049] In yet another embodiment, CM 1 46 may pro- 
vide an interface for direct connection to a communica- 
tion session hosted by media hub server 130 without 
need for a conference client. In this embodiment, CM 
146 presents a user interface that allows back-channel 
connection 1 26 to be utilized to return meeting summary 
content, current meeting status, participant information, 
shared data content, or even live conference audio. This 



might occur, for instance, if the participant has chosen 
not to use conference client 1 44 because the participant 
only wishes to monitor the activities of the communica- 
tion. It should be appreciated that the client component 

5 can be referred to as a thin client in that conference cli- 
ent 144 performs minimal data processing. For exam- 
ple, any suitable videoconference application can be 
conference client 144. As previously mentioned, CM 
146a is configured to recognize when the videoconfer- 

10 ence application of conference client A 1 44a starts and 
stops running, in turn, the CM can start and stop running 
as the conference client does. CM 1 46a can also receive 
information from the server component in parallel to the 
videoconference session. For example, CM 146a may 

15 allow participant A 122a to share an image during the 
conference session. Accordingly, the shared image may 
be provided to each of the client monitors so that each 
participant is enabled to view the image over a docu- 
ment viewer rather than through the video display region 

20 of the videoconference software. As a result, the partic- 
ipants can view a much clearer image of the shared doc- 
ument. In one embodiment, a document shared in a con- 
ference is available for viewing by each of the clients. 
[0050] The server component includes media hub 

25 server 130, which provides a multi-point control unit 
(MCU) that is configured to deliver participant custom- 
izable information. It should be appreciated that media 
hub server 130 and the components of the media hub 
server are software code configured to execute func- 

30 tionality as described herein. In one embodiment, media 
hub server 130 is a component of a hardware based 
server implementing the embodiments described here- 
in. Media hub server 130 includes media mixer 132, 
back-channel controller 140, and event handler 142. 

35 Media hub server 1 30 also provides conference connec- 
tion 138. More specifically, conference connection A 
138a completes the link allowing the peer-to-peer vide- 
oconferencing software of conference client A 144a to 
communicate with media hub server 130. That is, con- 

40 ferencing endpoint 138a emulates another peer and 
performs a handshake with conference client A 144a, 
which is expecting a peer-to-peer connection. In one 
embodiment, media hub server 1 30 provides Multipoint 
Control Unit (MCU) functionality by allowing connec- 
ts tions of separate participants into selectable logical 
rooms for shared conference communications. As an 
MCU, media hub server 130 acts as a "peer" to a con- 
ference client, but can also receive calls from multiple 
participants. One skilled in the art will appreciate that 

50 media hub server 1 30 internally links all the participants 
of the same logical room, defining a multi-participant 
conference session for each room, each peer-to-peer 
conference client operating with the media hub only as 
a peer. As mentioned above, media hub server 1 30 is 

55 configured to conform to the peer requirements of con- 
ference client 144. For example, if the conference cli- 
ents are using H.323 compliant conference protocols, 
as found in applications like MICROSOFTS NETMEET- 
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ING : media hub server 130 must also support the H.323 
protocol. Said another way, the conference communica- 
tion can occur via H.323 protocols, Session Initiated 
Protocols (SIP), or other suitable APIs that match the 
participant connection requirements. 
[0051] Still referring to Figure 4, media mixer 132 is 
configured to assemble audio and video information 
specific to each participant from the combination of all 
participants' audio and video, the specific participant 
configuration information, and server user-interface set- 
tings. Media mixer 132 performs multiplexing work by 
combining incoming data streams, i.e., audio/video 
streams, on a per participant basis. Video layout proc- 
essor 134 and audio distribution processor 136 assem- 
ble the conference signals and are explained in more 
detail below. Client monitor-back-channe! network al- 
lows media hub server 130 to monitor a user's interac- 
tions with conference client 144 and to provide the ap- 
pearance that the peer-to-peer software application has 
additional functionality. The additional functionality 
adapts the peer-to-peer functionality of the software ap- 
plication, executed by conference client 1 44, for the mu I- 
ti-participant environment described herein. The client 
monitor-back-channel network includes client monitor 
146 back-channel connection 126, back-channel con- 
troller 140, and event handler 142. 
[0052] Back-channel connection 1 26 is analogous to 
a parallel conference in addition to conference channel 
124. Back-channel controller (BCC) 140 maintains the 
communication link from each client monitor. Protocols 
defined on the link are interpreted at media hub server 
1 30 and passed to the appropriate destinations, i.e., oth- 
er participant's back-channel controllers, event handler 
142, or back to the CM 146. Each of the back-channel 
controllers 140 are in communication through back- 
channel controller communication link 148. 
[0053] In one embodiment, media hub server 130 pro- 
vides a client configurable video stream containing a 
scaled version of each of the conference participants. A 
participant's event handler 142 in media hub server 1 30 
is responsible for maintaining state information for each 
participant and passing this information to media mixer 
132 for construction of that participants user-interface. 
In another embodiment, a server-side user-interface 
may also be embedded into the participant's video/au- 
dio streams as will be explained in more detail below 
with reference to Figure 8. 

[0054] Figure 5 is a schematic diagram of the compo- 
nents for a multi-participant conference system using a 
client monitor back-channel wherein a non-participant 
can join the conference in accordance with one embod- 
iment of the invention. Non-participant connection 150 
is in communication with back-channel communication 
link 148. Here, a back-channel connection 128 can be 
established between non-participant client 150 and 
back-channel controllers 140 of media hub server 130. 
In one embodiment, back-channel communication link 
148 enables each of the back-channei controllers to 



communicate among themselves, thereby enabling cor- 
responding client monitors or non-participants to com- 
municate via respective back-channel connections 1 26. 
Accordingly, images and files can be shared among cli- 
5 ents over back-channel communication link 148 and 
back-channel connections 126. In addition, a non-par- 
ticipant back-channel connection can be used to gain 
access to media hub server 130 for query of server sta- 
tus, conference activity, attending participants, connec- 
10 tion information, etc., in one embodiment. Thus, the 
non-participant back-channel connection acts as a back 
door to the server or a conference session. From the 
server, the non-participant can obtain information for an 
administrator panel that displays conference and server 
15 performance, status, etc. From the conference session 
the non-participant can obtain limited conference con- 
tent across back-channel communication link 148, such 
as conference audio, text, images or other pertinent in- 
formation to an active conference session. 
20 [0055] Figure 6 is a high level schematic diagram of 
the media hub server in accordance with one embodi- 
ment of the invention. Media hub server 130 includes 
media mixer 132. Video layout processor 134 is includ- 
ed in media mixer 1 32. In one embodiment, video layout 
25 processor 134 is responsible for generating a composite 
video image for each participant by combining all other 
participant's video using the chosen video layout and 
participant configuration information defined by each 
participant through the client monitor-back-channel net- 
so work. A type of video layout ch osen by a participant may 
depend upon the conference setting or the number of 
participants. For example, a two-user communication 
may appear identically to a peer-to-peer connection, i. 
e., each participant fills the other's video window. Alter- 
35 natively, three or more users may present a tiled and 
configurable video display that will show only the other 
active members in a conference, i.e., a participant will 
not see his own video stream. Exemplary video layouts 
are described in more detail below with reference to Fig- 
40 ures12and13. 

[0056] Audio distribution processor 136 is also includ- 
ed in media mixer 1 32. As audio plays a key role in any 
conference environment, the ability to hear the speaker 
or each of the other participants is important. In a meet- 
45 ing/workgroup conference, each participant typically 
wishes to hear all other participants. However, in a pres- 
entation/training conference, the speaker wishes to only 
hear a questioner while the audience wishes to primarily 
hear the speaker and possibly the questioner. These 
50 various configurations are options provided by media 
hub server 130 through audio distribution processor 
136. In one embodiment, the audio options are extend- 
ed to include listening to the loudest participant, or loud- 
est group of participants, listening only to a single speak- 
55 er with the capability of logically "passing the micro- 
phone" to an appropriate participant. In addition, the log- 
ical "speaker" often becomes the primary video image 
distributed to the other participants, in another embod- 
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iment, an interface allowing a participant to create a pri- 
vate audio link to any other participant is enabled 
through audio distribution processor 136 ; as will be ex- 
plained further below. 

[0057] Transcoding 160 is included in media mixer 
1 32. Transcoding 1 60 enables the conversion of onefor- 
mat to another. Transcoding 160 generally performs 
functions that benefit the video and audio processing 
functions of the media mixer 132. One skilled in the art 
will understand that various transcoding methods need 
be used to perform video scaling, resolution and bit- 
depth conversions, media stream format conversions, 
adjustments for bitrate control, and other requirements. 
In one embodiment, transcoding may further result in 
more complete transformations. For example, an audio 
signal can be converted into text in one embodiment. 
The text can be supplied to a non-participant connec- 
tion, such as the non-participant connection of Figure 5. 
Session manager 164 is included in media hub server 
1 30 . Session manager 1 64 commu nicates with the com- 
ponents of connection manager 1 62 and supplies infor- 
mation to media mixer 132. Session manager 1 64 allo- 
cates and controls the logical rooms that group partici- 
pant conference connections, thereby identifying sepa- 
rate conference sessions on media hub server 130. In 
one embodiment, collaboration models maintained by 
session manager 164 define sets of rules that will gov- 
ern a given conference session and determine collabo- 
ration behavior. These rules are communicated to the 
media mixer 132 to adjust processing functions as de- 
scribed with reference to Figure 8. 
[0058] Connection manager 1 62 includes the confer- 
ence channel, the back-channel controller and the event 
handler for each participant. The parallel networks de- 
fined by the conference channel and the back-channel 
with reference to Figure 4 are processed through con- 
nection manager 162. Any suitable number of devices 
166a-166n for a multi-participant conference, commu- 
nicate with connection manager 162. As mentioned 
above, devices 166a-166n are thin clients in one em- 
bodiment of the invention. 

[0059] Figure 7 is a more detailed schematic diagram 
of the client monitor connection between the client and 
the media hub server in accordance with one embodi- 
ment of the invention. The client for participant A 122a 
includes conference client 144a and client monitor 
146a. Conference client 144a includes a peer-to-peer 
videoconferencing application having a graphical user 
interface (GUI) with a video display window 170. Addi- 
tionally, the GUI provides a number of buttons enabling 
functionality suitable for videoconferencing software, as 
well as display box 172 identifying the conference par- 
ticipants. As mentioned above client monitor 146a mon- 
itors events within display window 1 70. CM 1 46a estab- 
lishes back-channel connection 126a with media hub 
server 1 30. In one embodiment, when conference client 
144a establishes conference channel connection 124a 
with media hub server 130, CM 146a also places a call 



to establish back-channel connection 126a. Back-chan- 
nel connection 126a carries system information, such 
as user interface (Ul) events, status information, partic- 
ipants connected, etc. In one embodiment, back-chan- 

5 nel connection 126a is used as a control channel to 
change or define how the video and audio signals come 
across conference channel 1 24a. That is, the audio and 
video streams delivered to each client and how they are 
mixed are defined from the information provided from 

10 CM 146a over back-channel connection 126a. 

[0060] Still referring to Figure 7, media hub server 1 30 
includes connection manager 1 62 and media mixer 1 32. 
It should be appreciated that session manager 164 of 
Figure 6 is also included, although not shown here in 

'5 Figure 7. Connection manager 162 allocates compo- 
nents for each participant. For example, the compo- 
nents allocated to participant A includes conference 
connection 138a, back-channel controller 140a and 
event handler 142a for participant 122a. As discussed 

20 above, conference connection 138a acts as a confer- 
encing endpoint for conferencing client 144a. Back- 
channel controller 140a maintains the communication 
link from client monitor 146a. Event handler 142a proc- 
esses events from back-channel controller 1 40a. In one 

25 embodiment, event handler 142a maintains state infor- 
mation as necessary for processing of future events, for 
a respective participant. Event handler 142a communi- 
cates this information to media mixer 132, which in turn, 
configures the participant's user interface. The conf igu- 

30 ration of participant A's user interface is then transmitted 
through conference connection 138a and conference 
channel 124a to conference client 144a. 
[0061] CM 146a, while monitoring video display win- 
dow 1 70, may also define a user interface of which con- 

35 ference client 144a is a component along with a client 
user interface component. That is, CM 146a also in- 
cludes a module defining a user interface as discussed 
in more detail with reference to Figure 14. In one em- 
bodiment, CM 146a monitors the peer-to-peer applica- 

40 tion component and controls the client user interface. 
Here, further functionality can be provided through the 
client monitor in conjunction with the client monitor- 
back-channel network 1 48 connecting each of the client 
monitors as discussed with reference to Figure 14. It 

45 should be appreciated that the configuration of the com- 
ponents allocated by connection manager 1 62 is similar 
for each of the remaining participants 122b-122n, as 
compared to the components allocated to participant 
122a. Furthermore, each of participants 122a-122n are 

so interconnected through client monitor-back-channel 
network 148 through the respective back-channel con- 
trollers. 

[0062] Figure 8 is a schematic diagram of a video lay- 
out processor conf igu red to generate a composite video 
55 image for each participant in accordance with one em- 
bodiment of the invention. As mentioned previously, the 
type of video layout chosen may depend upon confer- 
ence settings or the number of participants. Video sig- 
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nais 172a-172e from five participants are supplied to 
video layout processor 134. Video layout processor 1 34 
combines the incoming video streams to be distributed 
to the conference participants according to a set of cri- 
teria. The set of criteria includes GUI criteria 178, user 
criteria 1 76 and model rules criteria 1 74. Thus, each par- 
ticipant is supplied a video layout consisting of portions 
of the input video streams in one embodiment. Each vid- 
eo layout 180a-180e is supplied back to the respective 
participant over the conference channel. For example, 
video layout 1 80a can be displayed in video display win- 
dow 170 of conference client 144a of Figure 7. Thus, 
the peer-to-peer application on the conference client is 
displaying a peer that looks like four people. 
[0063] Still referring to Figure 8, video layout 1 80a is 
configured as the video of participant C as a larger por- 
tion of the display window, with participant's B, D, and 
E occupying equal smaller areas. Region 182a is re- 
served to allow the media hub server to insert its own 
user interface directly into the outbound video stream 
image supplied to each participant. Region 1 82a is add- 
ed by media hub server as if it was a video display sim ilar 
to another participant. Region 182a can be filled with 
buttons, color patches, icons or other suitable images 
as determined by the server user- interface. For exam- 
ple, one server user-interface may show an icon, that 
when clicked, changes the layout of all the participants. 
In another example, a speaker may have an interface 
that prevents audio from all participants until a question- 
answer session begins. A user-interface icon shown 
through the region identified as the server user interface 
may be used to pass or request control from the current 
speaker to another participant, i.e., who will continue the 
conference. It should be appreciated that while region 
182a is described in particular as an interface that offers 
enhanced functionality to a participant, the same en- 
hanced functionality is offered to each participant 
through region 1 82. Since the client monitor is watching 
a participant's activity within the display window, activity 
within server user interface region 182a can be captured 
in order for some action to occur. It should be appreci- 
ated that the server is inserting video to appear as an 
interface and is not creating an operating system icon 
control to place on top of the video in the application 
layer. Consequently, the server component can dynam- 
ically modify the GUI element, GUI function and GUI el- 
ement location as directed by a user through the client 
monitor. 

[0064] The video-distributed server user interface dis- 
played through region 1 82a requires that the client mon- 
itor for participant A sends mouse actions, or other 
events, through the back-channel to the media hub serv- 
er. The media hub server can then process these events 
according to the participant's server-provided user inter- 
face, i.e. based upon event location in the video image. 
Since the user interface is sent within the video stream, 
any media hub server configuration can be done 
through the video window. For example, mouse events 



over the video image can be sent back to the server to 
control some aspect of the display. It should be appre- 
ciated that this feedback loop establishes a closed user 
interface for feature control. 

5 [0065] Any number of suitable layouts can be de- 
signed for video layouts 1 80a-1 80e as Figure 8 does not 
represent all possible layout options available. For ex- 
ample, server user interface (SUI) region 182 : or any 
other region, may be omitted or dynamically assigned. 

10 it should be appreciated that regions can be fixed or cus- 
tomizable. The server can have a fixed set of layouts, 
clients can utilize a defined protocol or language to de- 
fine a layout, or an external structure can be reported to 
the server that defines a layout. The conferencing pro- 
fs tocol between the conference client and the media hub 
server is used to negotiate the capabilities of the con- 
ference channel. The determined capabilities may fur- 
ther limit a participant's video layout options . One skilled 
in the art will appreciate that video and audio formats, 

20 video size, frame rates, and other attributes may be ne- 
gotiated based upon conference protocols, network 
bandwidth, latency and other criteria. 
[0066] In one embodiment, some participants may not 
have a video capture device, i.e., a camera, orthey may 

25 choose to have their respective video capture device 
turned off. However, the participants not having a video 
capture device are allowed to join a conference. Here 
an icon symbol representing the participant will be 
shown to the other conference members. This symbol 

30 allows other members to identify the participant visually 
and control their user-interface accordingly. The serv- 
er's media mixer will insert this icon into the video stream 
layout. Alternatively to the server providing default icons 
to be used for such participants, the back-channel con- 

35 nection can be utilized to deliver a custom participant 
icon from the participant's client monitor. The media mix- 
er will use this provided custom icon in place of the serv- 
er default. Where the participant does not have a video 
capture device, the participant can define the video dis- 

40 play the other participants receive by defining a pre-se- 
lected image. In some cases, participants may choose 
to use this pre-selected icon instead of their transmitted 
video stream. For example, the participant may wish to 
leave the conference for a moment, wish their video im- 

^5 age to remain anonymous, etc. The media hub server 
can accommodate such requests through instructions 
provided over the back-channel connection. 
[0067] Video layout processor 134 uses a set of cri- 
teria to determine how to mix the video signals. The set 

50 of criteria are represented by GU I criteria 1 78, user cri- 
teria 176 and model rules criteria 174. Model rules cri- 
teria 1 74 are determined by the collaboration model be- 
ing followed. For example, the collaboration models in- 
clude a one-to-one model, a one to many model, a group 

55 discussion model, etc. Accordingly, a group collabora- 
tion may have different model rules than a one to many 
collaboration. User criteria 176 is defined by the user 
among options available through the active session's 



19 EP 1 381 237 A2 20 

collaboration model. For example, a user may decide by the Video Layout Processor. It should be appreciated 
how to view multiple participants, i.e., how to configure that the client monitor is watching the video display win- 
the various regions such as video layout 180a-180e. dow ; therefore, the mouse activity is reported to the me- 
GUI criteria 178 includes the functionality enabled dia hub server through the back-channel. It will be ap- 
through server user interface region 182 discussed 5 parent to one skilled in the art that a participant can tar- 
above. In one embodiment, the set of criteria is arranged get his audio to one or more of the participants. For ex- 
in a hierarchical order, i.e., model rules criteria 1 74 limit ample, participant C 1 22c can target his audio to partic- 
user criteria 176, which in turn limit GUI criteria 178. ipant B 122b and participant N 122n to set up a private 
[0068] Figure 9 is a schematic diagram of the audio audio channel between the three participants. In anoth- 
distribution processor in accordance with one embodi- 10 er embodiment, the audio distribution processor adjusts 
ment of the invention. The ability to hear the speaker or the volume of the main speaker, participant A 122a, dur- 
each of the other participants is a core function of audio ing a sub-conference between participant B 122b and 
distribution processor 136. As is generally known vari- participant C 122c. As discussed above with reference 
ous collaboration models require different audio distri- to Figure 8, audio distribution processor 136 is subject 
bution. For example, a workgroup conference model 15 to similar set-up criteria as the video layout processor, 
has a different configuration than a training conference That is, the model rules criteria establish the rule of col- 
model as discussed above with reference to Figure 7. laboration, the user criteria establish a user's preferenc- 
For a training conference, each audience participant es within the model rules and the GUI criteria insert 
hears the speaker, and the speaker hears each audi- some audio signal into the conference. For example, the 
ence participant. It is not required that each audience 20 model rules may preclude sub-conferencing in one em- 
participant hear the audio from other participants until a bodiment. 

participant has a question. Audio signals from each of [0071] Figures 11A-11C are schematic diagrams of 
participants A-N 122a-122n is provided to audio distri- patterns for mixing audio streams in accordance with 
bution processor 136 over the conference channel. Par- one embodiment of the invention. Figure 11A shows a 
ticipant A 122a is provided with an audio signal from 25 matrix of four participants, A-D, where each participant 
each of the other participants. Of course, participant A is enabled to receive a signal from each of the other par- 
122a does not listen to its own audio signal. As men- ticipants. For example, participant A is enabled to re- 
tioned elsewhere, each participant may configure the ceive a signal from participants B,C and D. Participant 
volume of the audio signals and which signal is being B is enabled to receive a signal from participants A, C 
listened to. It should be appreciated that audio signals 30 and D and so on. Figure 11 B illustrates the matrix for a 
are transmitted across the conference channel. sub-conferencing audio link between participants A, C 
[0069] Figure 10 is a schematic diagram of the audio and D. Here, participant A has created a private audio 
distribution processor configured to provide private au- link with participants C and D. That is, participant B will 
dio communications in accordance with one embodi- not receive the audio signal being sent from A here. Fig- 
ment of the invention. The ability to create a private au- 35 ure 11C illustrates the resulting matrix when the sub- 
dio link allows an audience member to comment on the conferencing feature between participants A, C, and D 
conference with another participant without other partic- is activated. Here, participant B will not receive any sig- 
ipants hearing this communication. In such an instance, nal from participant A during the sub-conference. Addi- 
the Video Layout Processor may optionally stall the vid- tionally, during the sub-conference between participants 
eo images of the linked participants or even supply a 40 A, C and D, the volume for the audio from participant A 
pre-selected image during the private communication. to C and D is at 100% of the audio signal from participant 
For example, if participant A 1 22a is speaking, partici- A, while the volume for the remainder of the participants 
pant C 122c can have a private conversation with par- being received by C and D is set at 50 %. Of course, 
ticipant B 122b, where intra-meeting audio channel 1 84 any suitable percentages of volume can be used here 
is created between participant B and participant C 45 to allow a participant to hear the audio from the person 
through audio distribution processor 136. initiating the sub-conference. For example, the volume 
[0070] In one embodiment, intra-meeting audio chan- of the other participants can drop to zero (0) in one em- 
nel 1 84 between two participants is constructed by one bodiment. 

participant's mouse pointer being held over the video [0072] Continuing with sub-conferencing example 
image of the other participant in a video layout on the 50 above, the sub-conference initiated by participant A can 
conference client and then holding the mouse button be configured as a one-way audio path or as a two-way 
down. Thus, participant C 1 22c holds his mouse pointer audio path. That is, in one embodiment participant A*s 
over the image of participant B 122b to create the intra- action of initiating a sub-conference between partici- 
meeting audio channel. The connection remains while pants C and D does not effect the control of participants 
the mouse button is in the down state. In one embodi- 55 c and D of their own audio. Thus, participants C and D 
ment, the receiving participant will see a video cue that must use the mouse-down interface if they want to corn- 
can be used to determine who is speaking privately with ment back to selected participants, as participant A has 
him. This video cue is inserted into the video streams done for the sub-conference. In another embodiment, 
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participant A's initiation of the sub-conference with par- 
ticipants C and D creates communication links as if par- 
ticipant C selected a private link with participants A and 
D and as if participant D selected a private link with par- 
ticipants A and C. Thus, participant A's action blocks the 
audio from participants C and D from being heard by 
other participants, i.e., participant B. 
[0073] Figure 12 is a schematic diagram of the effect 
of an event on a conference client's video display win- 
dow in accordance with one embodiment of the inven- 
tion. Example video layout 188 is configured such that 
a primary participant video is in region R1 while other 
participants are located in regions R2, R3 and R5. Re- 
gion R4 contains the server user interface (SUI) as dis- 
cussed above. More specifically, participant B's video 
layout can be configured with participant A in the primary 
region and participants C, D, and E in the secondary 
regions as in video layout 190. If participant B clicks the 
mouse while the pointer is over the region displaying 
participant E, then participant E will be moved to the pri- 
mary region and participant A is moved from the primary 
region to the region previously occupied by participant 
E, as illustrated in video layout 192. Even conference 
video can be thought of as a GUI element and modified 
similarly. For example, clicking on a participant's video 
region can result in a change in brightness of the image 
sent by the server component. 

[0074] Figure 13 is a schematic diagram of another 
effect of an event on a conference client's video display 
window in accordance with one embodiment of the in- 
vention. Here, a participant double clicks on participant 
C of video layout 1 90. The double-click event results in 
video layout 194 where the image of participant C oc- 
cupies the entire video display region. Furthermore, 
double-clicking the mouse while the pointer is over the 
display of participant C will return the image to video 
layout 190. It should be appreciated that any suitable 
number of events can be defined to allow a participant 
to configure the video display region. For example, as 
mentioned above, by clicking and holding the mouse 
button over a video of a participant on the video display 
layout will establish an audio connection with that par- 
ticipant. Thus, a private audio link for a sub-conference 
can be created. As with other common application inter- 
faces, this list of events can be extended to include a 
particular mouse button (i.e. Left, Middle, Right) and any 
keyboard state information at the time of mouse activity 
(i.e. Shift-Key pressed, Ctrl-Key pressed, etc.). Other 
events including a mouse movement tracking and key- 
strokes may also be defined. In one embodiment, a 
server interface may provide a region in the video layout 
that is shown to audience participants in a training con- 
ference. When clicked by a participant, indicating that 
the participant has a question, the speaker's user-inter- 
face may show a visual cue to identify the member with 
the question. In response, the speaker could have an 
interface to manage a virtual "microphone", allowing the 
participant the floor the question, yet retain the ability to 



capture the microphone back for conference continua- 
tion. 

[0075] The back-channel is not reserved only for serv- 
er configuration and user-interface protocols. It can also 
5 be used as a communication channel between partici- 
pants. Client monitors can communicate among them- 
selves by sharing and exchanging information on the 
back-channel through the media hub server. For exam- 
ple, the client monitor may wish to present a separate 
w user-interface in parallel to that provided by the confer- 
ence client. In one embodiment, the client monitor could 
capture the application window of a POWERPOINT ap- 
plication on the participant's computer. This information 
could be transmitted, say as a JPEG image, to the other 
is client monitors where it would be displayed. In this way, 
a participant could share a high-resolution slide image 
of his presentation with all other participants without re- 
lying solely on the small resolution of an attached video 
capture device. 
20 [0076] Conference content information, summary no- 
tations, chat, or other connection status information can 
be relayed among the participants on the back-channel. 
In one embodiment, a specialized protocol to the media 
hub server allows for reporting activity and membership 
25 of participants to a conference. As with the example 
mentioned above, the system displays shared JPEG im- 
ages on each client's machine in a resizable window. 
The received images can be scaled based upon window 
size or viewed according to actual pixel resolution using 
30 scrollbars. 

[0077] Figure 14 is a schematic diagram of a client 
monitor graphical user interface which includes the user 
interface provided by the conference client in accord- 
ance with one embodiment of the invention. Client mon- 
35 itor GUI 200 includes conference client application win- 
dow GUI 202 and client monitor user interface 204. In 
one embodiment, conference client application window 
GUI 202 is brought in as a component of client monitor 
GUI 200. That is, the code of the peer-to-peer applica- 
nt? tion is running GUI 202. It should be appreciated that 
GUI 202 is another representation of the GUI for con- 
ference client 1 44a of Figure 7. Client user interface 204 
allows for enhanced functionality to occur through the 
back-channel. For example, files, documents, images, 
45 etc. can be sent to other client monitors across the back- 
channel to be displayed in document viewer region 206 
associated with that client monitor. In particular, a POW- 
ERPOINT presentation that a speaker is discussing 
may be viewed by each of the participants. It should be 
50 appreciated that GUI 200 can be opened up with the 
p eer _to-peer application being acomponent of GUI 200. 
Alternatively, the peer-to-peer application can be 
opened up and when enhanced functionality is required 
another GUI is opened up. It will be apparent to one 
55 skilled in the art that any suitable navigation tool, such 
as scroll bars, drop down menus, tabs, icons, buttons, 
etc. can be used to provide the options for a participant 
to choose from the offered functionality. 
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[0078] Client user interface 204 also includes partici- 
pants* region 208 listing the participants of the confer- 
ence. Files associated with a particular participant can 
be listed as is shown with respect to participant 1 of par- 
ticipants' region 208. Local files region 210 includes files 
that can be shared between participants. Devices' re- 
gion 212 provides remote devices configured to supply 
information for the conference for a particular client. For 
example, a scanner in communication with the respec- 
tive client can be used to scan documents so that the 
participants can share the documents. A second docu- 
ment viewer region 214 is included to view a document 
in shared space. Additionally, a document being 
scanned from the scanning device listed in region 212 
can be viewed in region 214. Thus, as a document is 
being scanned, the participant can view the document 
in region 214. Conference log region 216 provides a run- 
ning log of participants joining the conference and the 
time at which the participant joined. It should be appre- 
ciated that the conference log could record other suita- 
ble items such as when participants signed off. Spare 
region 218 can be used to provide any further suitable 
user interface for the videoconference environment. It 
should be appreciated that any number of suitable con- 
figurations can be supplied for GUI 200. In one embod- 
iment, the back-channel controller allows the server to 
distribute the documents between clients, similar to the 
distribution of video and audio signals over the back- 
channel network. 

[0079] In one embodiment, a user can download the 
client monitor over a distributed network. Here, the user 
can then utilize a server managed by an application 
service provider or a server on a local network allowing 
conferencing within an organization or division of a large 
corporation. Additionally, the code enabling the func- 
tionality described herein can be incorporated into 
firmware of devices used for videoconferencing, such 
as video projectors. Accordingly, the images from the 
projector can be supplied through the back-channel to 
participants of the conference. 

[0080] Figure 15 is a flowchart diagram of the method 
operations for creating a multi-user conferencing envi- 
ronment between conference clients having peer-to- 
peer conferencing applications in accordance with one 
embodiment of the invention. The method initiates with 
operation 220 where a server component is provided. 
In one embodiment, the server component is configured 
to emulate a peer-to-peer connection for each of the 
conference clients, One suitable server component is 
the media hub server component described above. The 
method then advances to operation 222 where a con- 
ference channel is defined for communication between 
conference clients and the server component The con- 
ference channel is configured to provide real time audio 
and video data in one embodiment. In another embod- 
iment, the conference channel is configured to support 
a conferencing protocol such as the H.323 protocol and 
the SIP protocol. 



[0081] The method of Figure 15 then proceeds to op- 
eration 224 where activities of a user in an active region 
are monitored. Here, a client monitor can monitor the 
video display region as described above. The activities 

5 being monitored include mouse activities of a user in the 
video display region. The method then moves to oper- 
ation 226 where an active selection of a user in the ac- 
tive region is reported. As described with reference to 
Figures 12 and 13 a user can click on a region of the 

10 video layout of the display window. The active selection, 
i.e., mouse click, is reported to the server component by 
the client monitor over the back-channel in parallel to 
the conference session being transmitted over the con- 
ference channel. The method then advances to opera- 

15 tion 228 where the configuration of an audio/video signal 
being supplied to a conference client associated with the 
user is modified, in response to the active selection re- 
porting being received by the server component. For ex- 
ample, the video display window can be modified here 

20 as discussed above with reference to Figure 12. 

[0082] In summary, the above described invention 
provides a videoconferencing system having enhanced 
functionality through a back-channel network. The sys- 
tem takes a pre-existing peer-to-peer application and 

25 provides a conference connection so that the applica- 
tion sees a peer-to-peer connection, however, in reality 
audio and video signals from multiple participants are 
being provided. The back-channel network acts as a 
parallel network to the conference channel. A client 

30 monitor watches a display window of the peer-to-peer 
application for user events, such as mouse oriented op- 
erations. Data captured by the client monitor is provided 
overthe back-channel to a media hub server. The media 
hub server responds to the data by modifying or config- 

35 uring the video and audio signals supplied to each par- 
ticipant over the conference channel. The conference 
system is configured to be joined by other non-partici- 
pants through the back-channel network. In addition, the 
back-channel allows for files to be shared between par- 

40 ticipants through a client interface defined and control- 
led through the client monitor. In one embodiment, a pe- 
ripheral client device, such as a scanner is enabled to 
scan a document into the system so that the document 
can be provided to each by the back-channel network. 

45 The document can be viewed by each client through the 
client interface. 

[0083] With the above embodiments in mind, it should 
be understood that the invention may employ various 
computer-implemented operations involving data 

so stored in computer systems. These operations are 
those requiring physical manipulation of physical quan- 
tities. Usually, though not necessarily, these quantities 
take the form of electrical or magnetic signals capable 
of being stored, transferred, combined, compared, and 

55 otherwise manipulated. Further, the manipulations per- 
formed are often referred to in terms, such as producing, 
identifying, determining, or comparing. 
[0084] The invention can also be embodied as com- 
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puter readable code on a computer readable medium. 
The computer readable medium is any data storage de- 
vice that can store data which can be thereafter read by 
a computersystem. Examples of the computer readable 
medium include hard drives, network attached storage 
(NAS), read-only memory, random-access memory, 
CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other 
optical and non-optical data storage devices. The com- 
puter readable medium can also be distributed over a 
network coupled computer systems so that the compu- 
ter readable code is stored and executed in a distributed 
fashion. 

[0085] Although the foregoing invention has been de- 
scribed in some detail for purposes of clarity of under- 
standing, it will be apparent that certain changes and 
modifications may be practiced within the scope of the 
appended claims. Accordingly, the present embodi- 
ments are to be considered as illustrative and not re- 
strictive, and the invention is not to be limited to the de- 
tails given herein, but may be modified within the scope 
and equivalents of the appended claims. 

Claims 

1 . A videoconference system, comprising: 

a client component having a monitoring agent 
configured to detect events within a video dis- 
play window of the client component; 
a server component configured to distribute 
video and audio data streams to participants of 
a conference session; 

a conference channel communication connec- 
tion over which the video and audio data 
streams are carried between the client compo- 
nent and the server component; and 
a back-channel communication connection 
over which events captured by the monitoring 
agent are transmitted to the server component, 
wherein the back-channel communication con- 
nection enables each of the participants to de- 
fine a video layout of the video display window. 

2. The videoconference system of claim 1, wherein 
the back-channel communication connection ena- 
bles each of the participants to communicate with 
other participants without disturbing the conference 
session. 

3. The videoconference system of claim 1, wherein 
the back-channel communication connection ena- 
bles each of the participants to communicate with a 
non-participant without disturbing the conference 
session. 

4. The videoconference system of claim 1, wherein 
the back-channel communication connection is 



configured to accommodate a private audio link be- 
tween two of the participants, the private audio link 
being established in response to the monitoring 
agent detecting an event. 

5 

5. The videoconference system of claim 4, wherein 
the event is maintaining a mouse button in a down 
position while a mouse pointer associated with the 
mouse button is within a region of the video display 

10 window. 

6. The videoconference system of claim 5, wherein 
the region is one of a video image of a participant 
or a GUI element. 

15 

7. The videoconference system of claim 1, wherein 
the events include one of a mouse activity and a 
keyboard activity, both the mouse activity and the 
keyboard activity occurring while a pointer associ- 
ated with the mouse activity or the keyboard activity 
is over a region of the video display window. 

8. A back-channel communication network for a vide- 
oconferencing system for a conference between a 
plurality of participants, comprising: 

a monitoring agent associated with a client, the 
client configured to execute a peer-to-peer vid- 
eoconferencing application, the monitoring 
agent monitoring a video display window con- 
trolled by the peer-to-peer conferencing appli- 
cation; 

a back-channel controller in communication 
with the monitoring agent over a back-channel 
connection, the back-channel controller config- 
ured to enable communication between the cli- 
ent and a plurality of conference clients over a 
back-channel controller communication link; 
and 

an event handler configured to enable insertion 
of server user interface data into an outbound 
video stream image for the client. 

9. The back-channel communication network of claim 
8, wherein the back-channel controller and the 
event handler are associated with a server compo- 
nent. 

10. The back-channel communication network of claim 
8, wherein the back-channel controller enables dis- 
tribution of files between the plurality of participants 
during a conference session. 

11 . The back-channel communication network of claim 
8, wherein the event handler maintains state infor- 
mation for each of the plurality of participants. 

12. The back-channel communication network of claim 
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11 , wherein the event handler provides the state in- 
formation to a media mixer for construction of a us- 
er-interface of the client. 

13. The back-channel communication network of claim 

12, wherein the user-interface of the client includes 
a server user-interface region, the server user-in- 
terface region being video inserted to appear as an 
interface. 

14. The back-channel communication network of claim 
8, wherein the event handler defines a video layout 
of the video display window of the client. 

15. The back-channel communication network of claim 
1 2, wherein the user interface of the client is defined 
within the video display window. 

16. A method for enhancing conference content deliv- 
ery for a videoconference session between multiple 
participants, comprising 

monitoring a video display window associated 
with a client; 

establishing a conference channel connec- 
tion for transmitting a video stream and an audio 
stream between the client and a server; 

detecting the establishment of the conference 
channel connection; 

in response to detecting the conference chan- 
nel connection, the method includes, 

establishing a back-channel connection be- 
tween the client and the server; 

displaying the video stream in the video dis- 
play window of the client; 

detecting an active selection in an active re- 
gion of the video display window; 

communicating the active selection to the 
server over the back-channel connection; 

modifying a configuration of one of the video 
stream and the audio stream at the server; and 

providing the modified configuration to the cli- 
ent over the conference channel connection. 

17. The method of claim 16, further including, 

inserting a server user-interface into the video 
stream; 

18. The method of claim 16, wherein the method oper- 
ation of establishing a back-channel connection be- 
tween the client and the server is transparent to a 
participant. 

19. The method of claim 16, wherein the active selec- 
tion is one of a mouse action and a keyboard mod- 
ifier. 

20. The method of claim 17, wherein the method oper- 
ation of inserting a server user-interface into the vid- 



eo stream is enabled by an event handler providing 
data to a media mixer over a back-channel network 
that includes the back-channel connection. 

5 21. A method for providing participant customizable 
video and audio streams for a videoconference ses- 
sion between a plurality of participants, comprising: 

providing a plurality of clients, each of the plu- 
10 rality of clients associated with a participant; 

providing a server in communication with the 
plurality of clients; 

establishing a first communication channel and 
second communication channel between the 
15 server and each of the plurality of clients, the 

first communication channel providing audio/ 
video data, the second communication channel 
providing system information; 
monitoring a video display window of a client; 
20 and 

providing feedback from the monitoring of the 
video display window over the second commu- 
nication channel to modify the audio/video data 
being supplied over the first communication 
25 channel. 

22. The method of claim 21, wherein the server in- 
cludes a media hub server component. 

30 23. The method of claim 21 , wherein each of the plural- 
ity of clients participates in the videoconference 
session through a peer-to-peer videoconference 
application. 

35 24. The method of claim 23, wherein the server pro- 
vides a conference connection for each of the plu- 
rality of clients, the conference connection config- 
ured to emulate a peer. 

40 25. The method of claim 21 , wherein the method oper- 
ation of monitoring a video display window of a cli- 
ent is performed through an external client monitor. 

26. The method of claim 21 , wherein the feedback in- 
45 eludes configuration preferences for a video layout 

for a participant associated with the client. 

27. The method of claim 21, wherein the feedback is 
provided through an external client monitor config- 

50 ured to watch the video display window of the client. 

28. A computer readable media having program in- 
structions for providing participant customizable 
video and audio streams for a videoconference ses- 

55 sion between a plurality of participants, comprising: 

program instructions for providing a plurality of 
clients, each of the plurality of clients associat- 
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ed with a participant; 

program instructions for providing a server in 
communication with the plurality of clients; 
program instructions for establishing a first 
communication channel and second communi- 
cation channel between the server and each of 
the plurality of clients, the first communication 
channel providing audio/video data, the second 
communication channel providing system infor- 
mation; 

program instructions for monitoring a video dis- 
play window of a client; and 
program instructions for providing feedback 
from the monitoring of the display window over 
the second communication channel to modify 
the audio/video data being supplied over the 
first communication channel. 

29. The computer readable media of claim 28, wherein 
the server includes a media hub server component. 

30. The computer readable media of claim 28, wherein 
the second communication channel is between an 
external client monitor and a back-channel control- 
ler of the server. 

31. The computer readable media of claim 30, wherein 
the external client monitor is configured to monitor 
the video display window of the client. 

32. The computer readable media of claim 28, further 
including: 

program instructions for enabling a private au- 
dio link over the second communication chan- 
nel, the private audio link defined between two 
participants during a videoconference session. 

33. A videoconferencing system configured to utilize 
peer-to-peer videoconferencing software to provide 
a multi-participant conference environment for a 
plurality of participants, comprising: 

a client component, the client component in- 
cluding, 

a conference client enabled to execute 
peer-to-peer videoconferencing software, the 
conference client communicating video and au- 
dio data across a conference channel; and 

a client monitor configured to monitor 
both, whether the conference channel is active 
and events within a video window displayed by 
the conference client, wherein the events within 
the video window are communicated across a 
back-channel connection, the back-channel 
connection established when the conference 
channel is active; 

a server component, the server component 



having a back-channel controller in communi- 
cation with the client monitor through the back- 
channel connection, the server component pro- 
viding a client configurable audio/video stream 
5 for each of a plurality of participants. 

34. The videoconferencing system of claim 33, wherein 
the client monitor defines a graphical user interface 
of which the video window displayed by the confer- 

10 ence client is a component. 

35. The videoconferencing system of claim 34, wherein 
the graphical user interface enables access to files 
of the conference client. 

15 

36. The videoconferencing system of claim 35, wherein 
the files of the conference client are available to 
each of the plurality of participants over the back- 
channel connection. 

20 

37. The videoconferencing system of claim 33, wherein 
the server component includes a media mixer con- 
figured to compose a composite audio/video signal 
for each of the plurality of participants from individ- 

25 ual audio/video signal from each of the plurality of 
participants. 

38. A videoconferencing system, comprising: 

30 a client component including a client in commu- 

nication with a client monitor; 
a server component; 

a conference channel defined between the cli- 
ent component and the server component, the 

35 conference channel providing a first path for re- 

al-time video/audio data to be exchanged be- 
tween the client component and a conferencing 
endpoint of the server component for a video- 
conference; and 

40 a back-channel defined between the client 

component and the server component provid- 
ing a second path for system information to be 
exchanged between the client monitor and the 
server component. 

45 

39. The videoconferencing system of claim 38, wherein 
the client includes a peer-to-peer videoconferenc- 
ing application. 

50 40. The videoconferencing system of claim 39, wherein 
the client monitor is configured to detect an activity 
in a display window associated with the peer-to- 
peer videoconferencing application, in response to 
detecting the activity, the client monitor reports the 

55 activity to the server component over the back- 
channel. 

41 . The videoconferencing system of claim 40, wherein 
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the activity is one of mouse movement, mouse 
clicks and keyboard state information. 

42. The videoconferencing system of claim 38, wherein 
the client monitor is configured to provide a user in- 
terface, the user interface including a display win- 
dow of a peer-to-peer videoconference application 
associated with the client 

43. The videoconferencing system of claim 38, wherein 
the server component is configured to enable ac- 
cess to a non-participant of the videoconference 
through a back-channel network associated with 
the back-channel. 

44. The videoconferencing system of claim 38, wherein 
the server component includes, 

a media mixer enabling distribution of a com- 
posite audio/video data stream to the client compo- 
nent, the media mixer in communication with a 
back-channel network to enable a private audio link 
between two clients. 

45. The videoconferencing system of claim 38, wherein 
the system information includes a configuration of 
a video display window associated with the client. 

46. The videoconferencing system of claim 45, wherein 
the system information is communicated to a media 
mixer of the server component in response to re- 
ceiving the system information, the media mixer 
modifies a video data stream for the client. 

47. A conferencing system configured to provide a mul- 
ti-user conference environment to deliver custom- 
izable information to a plurality of conference cli- 
ents, comprising: 

a client component, the client component in- 
cluding, 

a conference client; and 

a client monitor configured to monitor an 
activity of the conference client, the activity oc- 
curring over a video frame displayed by the 
conference client; 

a server component, the server component in- 
cluding, 

a media hub server component providing 
a conference connection, the media hub server 
component including, 

a media mixer configured to assem- 
ble audio and video data to be supplied to the 
conference client from audio and video data re- 
ceived by 

the media mixer from a plurality of confer- 
ence clients, the media mixer including, 

a video layout processor configured 
to generate a composite video image for each 



of the plurality of conference clients, and 

an audio distribution processor for 
providing an audio signal for each of the plural- 
ity of conference clients; 
5 a connection manager allowing con- 

nections of several participants into logical 
rooms for shared conference communications, 
the connection manager including, 

a back-channel controller enabling 
10 communication between the client monitor and 

the media hub server component, and 

an event handler configured to insert in- 
terface data into an outbound video stream im- 
age through the video layout processor. 

15 

48. The conferencing system of claim 47, wherein the 
interface data enables the conference client to ac- 
cess local files to be shared with the plurality of con- 
ference clients, the local files associated with a 

20 computer included in the client component 

49. The conferencing system of claim 47, wherein the 
client component and the server component are in 
communication through a conference channel car- 

25 rying real time audio/video data and a back-channel 
carrying system information; 

50. The conferencing system of claim 47, wherein the 
conference client includes, 

30 a peer-to-peer videoconference application in 

communication with the conference connection of 
the media hub server component. 

51. A graphical user interface (GUI) for a videoconfer- 
35 ence rendered on a computer monitor, comprising: 

a first region defining an integrated video com- 
ponent, the integrated video component asso- 
ciated with a client, the integrated video com- 
40 ponent having a plurality of participant video 

images, the integrated video component being 
monitored to detect user activity within a dis- 
play window of the integrated video compo- 
nent; and 

45 a second region providing access to files of a 

computer system; the second region allowing 
a user to select one of the files for transmission 
to a server supporting the videoconference, 
wherein the server communicates the selected 

so one of the files to participants of the videocon- 

ference. 

52. The GUI of daim 51 , wherein the user activity is one 
of mouse movement, mouse clicks and keyboard 

55 state information. 

53. The GUI of claim 51 , wherein the first region is as- 
sociated with a peer-to-peer videoconferencing ap- 
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plication. 

54. The GUI of claim 51 , wherein the second region en- 
ables a peripheral device to augment conference 
content viewable by the participants. 

55. The GUI of claim 54, wherein the peripheral device 
is one of a scanner and a video projector. 

56. The GUI of claim 51 , wherein the integrated video 
component is provided over a first communication 
link with the server and information captured in the 
second region is provided to the server over a sec- 
ond communication link. 

57. The GUI of claim 56 wherein the first communica- 
tion link is a conference channel and the second 
communication link is a back-channel. 

58. A method for providing a multi-user conference en- 
vironment for multiple participants, comprising: 

establishing a server component for enabling a 
conference channel connection between the 
server component and a conference client as- 
sociated with a participant; 
providing audio and video data from the partic- 
ipant to the server component over the confer- 
ence channel connection; 
communicating system preferences to the 
server component for each of the multiple cli- 
ents over a back-channel connection; 
distributing combined audio and video data to 
the participant over the conference channel 
connection, the combined audio and video data 
presented as defined by the system preferenc- 
es; 

monitoring an interaction of the participant with 
a video image presented on the conference cli- 
ent; 

transmitting a signal indicating the interaction 
to the server component over the back-channel 
connection; and 

in response to the signal indicating the interac- 
tion, modifying the combined audio and video 
data distributed to the conference client over 
the conference channel connection. 

59. The method of claim 58, wherein the conference 
channel connection support one of H.323 protocol 
and session initiation protocol (SIP). 

60. The method of claim 58, wherein the system pref- 
erences include one of a position of an image in a 
video layoutforeach of the multiple clients, abright- 
ness of the video layout and a volume level associ- 
ated with participants displayed in the video layout. 
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61. The method of claim 58, wherein the interaction is 
associated with one of a mouse movement and a 
keyboard signal. 

5 62. A method for creating a multi-user conferencing en- 
vironment between conference clients having peer- 
to-peer conferencing applications, comprising: 

providing a server component configured to 
to emulate a peer-to-peer connection for each of 

the conference clients; 

defining a conference channel for communica- 
tion between conference clients and the server 
component; 

*5 monitoring activities of a user in an active re- 

gion of a video display associated with one of 
the conference clients; 

reporting an active selection by a user in the 
active region to the server component, the re- 
20 porting occurring outside of the conference 

channel; and 

in response to the active selection reporting be- 
ing received by the server component, modify- 
ing a configuration of an audio/video signal pro- 
25 vided to the conference clients. 

63. The method of claim 62, wherein the server com- 
ponent is a media hub server. 

30 64. The method of claim 62, wherein the conference 
channel is configured to communicate real time au- 
dio and video data between the conference clients 
and the server component. 

35 65. The method of claim 62, wherein the method oper- 
ation of reporting an active selection by a user oc- 
curs over a back-channel. 

66. The method of claim 65, wherein the back-channel 
40 defines a communication link between a client mon- 
itor configured to track activities in a video display 
window of one of the conference clients and a back- 
channel controller of the server component. 

45 67. A computer readable media having program in- 
structions for creating a multi-user conferencing en- 
vironment between conference clients having peer- 
to-peer conferencing applications and a server 
component configured to emulate a peer-to-peer 

so connection for each of the participants, comprising: 

program instructions for defining a conference 
channel for communication between confer- 
ence clients and the server component; 
55 program instructions for monitoring activities of 

a user with one of the conference clients; 
program instructions for reporting the moni- 
tored activities to the server component over a 
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back-channel connection; and 
program instructions for modifying a video and 
audio signal provided to the conference clients 
in response to the reported activities being re- 
ceived by the server component. 5 

68. The computer readable media of claim 67, wherein 
the server component is a media hub server. 

69. The computer readable media of claim 67, wherein 10 
the back-channel connection defines a communica- 
tion link between a client monitor configured to track 
activities in a video display window of one of the 
conference clients and a back-channel controller of 

the server component. »5 

70. The computer readable media of claim 67, further 
including: 

providing program instructions for enabling a 20 
private audio link between two participants dur- 
ing a videoconference session. 
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